Memory technology has become so ubiquitous and low-cost that it is often offered free to users. For instance, well-known web sites provide free e-mail accounts to their users where large blocks of memory support the underlying accounts. To support such e-mail services and others, large server farms provide the necessary memory infrastructure (e.g., processors and servers) to manage the respective accounts. Moreover, memory storage itself has become more sophisticated. Generally, large memory subsystems are managed as a hierarchy where essential data being presently processed in a program is pulled into high-speed memory for further processing while lesser-used data remains on the permanent storage medium. The permanent storage medium is generally processed as a homogenous block of memory and which can create other problems by modeling the memory in such a manner. Before considering such problems, a discussion of the memory hierarchy is considered.
The main aspect of the memory hierarchy is to allow reasonably fast access to large amounts of memory. If only a little memory was necessary, fast, static RAM could be employed for all applications. If speed was not also necessary, low-cost dynamic RAM could be utilized to reduce costs. Thus, the memory hierarchy model takes advantage of the principle of locality of data reference to move often-referenced data into fast memory and leave less-used data in slower memory. Unfortunately, the selection of often-used versus lesser-used data varies over the execution of any given program. Therefore, designers cannot simply place data at various levels in the memory hierarchy and leave the data alone throughout the execution of the program. Instead, the memory subsystems need to be able to move data between themselves dynamically to adjust for changes in locality of reference during the program's execution.
The program is largely unaware of the memory hierarchy. In fact, the program only explicitly controls access to main memory and those components of the memory hierarchy at the file storage level and below (since manipulating files is a program-specific operation). In particular, cache access and virtual memory operation are generally transparent to the program. That is, access to these levels of the memory hierarchy usually takes place without any intervention by the program. Thus, the program merely accesses main memory and the hardware (and operating system) take care of the rest.
If the program really accessed main memory on each access, the program would run quite slowly since modern DRAM main memory subsystems are much slower than the CPU. The job of the cache memory subsystems (and the cache controller) is to move data between main memory and the cache so that the CPU can quickly access data in the cache. Likewise, if data is not available in main memory, but is available in slower virtual memory, the virtual memory subsystem is responsible for moving the data from hard disk to main memory (and then the caching subsystem may move the data from main memory to cache for even faster access by the CPU).
As noted above, the permanent memory storage subsystem is often viewed as a homogenous block of memory by the memory hierarchy model. While such a model may be easier to implement, it trades-off the ability to control power consumption in the system. Unfortunately, as memory subsystems have increased in size and density, the aggregate power consumption of the permanent memory storage subsystem has increased. Thus, by processing data as a contiguous or homogenous block of memory, power is inefficiently managed and consequently, overall system costs are increased.